Improve evaluate command docs with better examples and dataset guidance#81
Merged
Conversation
… help Mozilla pulled the Common Voice corpora off the Hugging Face Hub in October 2025 (now exclusively on Mozilla Data Collective), so the `mozilla-foundation/common_voice_17_0` repo is empty and the datasets server answers every request with HTTP 500 "The dataset is empty", breaking the subset/split example verbatim. Swap it for openslr/librispeech_asr --subset clean, verified end-to-end against the datasets-server rows API (ungated, parquet-backed, multiple subsets so --subset is genuinely required, auto-picked test split, auto-detected audio/text columns). Also expand the eval help with how scoring works (normalization, pooled summary) and a sampling of vetted datasets — peoples_speech, minds14, simsamu — plus two new examples covering non-English audio and HF diarization sets. https://claude.ai/code/session_01WpRHsDXCWYTffZ8eMGK6Er
alexkroman
enabled auto-merge (squash)
June 11, 2026 23:25
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Enhance the
evaluatecommand's help text and examples to provide clearer guidance on usage and available datasets.Changes:
openslr/librispeech_asr(English; clean/other subsets)MLCommons/peoples_speech(real-world US English)distil-whisper/meanwhile(long-form English)PolyAI/minds14(14 locales; banking calls)diarizers-community/simsamu(French dispatch calls with speaker turns)mozilla-foundation/common_voice_17_0) withopenslr/librispeech_asrfor better clarityPolyAI/minds14and--language-codediarizers-community/simsamuand--speaker-labelsThe changes make the command more discoverable and help users understand both what the command does and which datasets are suitable for evaluation.
https://claude.ai/code/session_01WpRHsDXCWYTffZ8eMGK6Er